Information processing apparatus, information processing method, and program

ABSTRACT

An information processing apparatus includes an image pickup unit, an extracting unit, an estimating unit, a categorizing unit, and a recording control unit. The image pickup unit captures an image of a subject. The extracting unit extracts a human figure from the captured image of the subject captured by the image pickup unit. The estimating unit estimates a posture of the human figure extracted by the extracting unit. The categorizing unit categorizes the posture of the human figure estimated by the estimating unit into a previously prepared pose. The recording control unit controls recording of the captured image on the basis of the pose that the posture of the human figure is categorized into by the categorizing unit.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to information processing apparatuses,information processing methods, and programs. More specifically, thepresent invention relates to an information processing apparatus, aninformation processing method, and a program allowing users to moresurely and more easily take images of a subject in a specific pose.

2. Description of the Related Art

Recently, it has been studied to automatically record an image of asubject in accordance with a pose of a person serving as the subjectwhile the image of the person is being captured withimage-capturing-function attached electronic devices, such as digitalcameras and mobile phones.

For example, there is a related art for comparing a feature point of askin color region extracted from a captured image with a feature pointof a pose image set in advance and recording the image if thecorresponding feature points match or are similar (see, for example,Japanese Unexamined Patent Application Publication No. 2008-263422).

SUMMARY OF THE INVENTION

A method disclosed in Japanese Unexamined Patent Application PublicationNo. 2008-263422 allows image capturing to be performed after recognitionof a shape of hand, such as a victory sign, based on a feature point ofa skin color region. However, capturing images after recognition of apose of an entire body is difficult with the method unless the skincolor region can be extracted regarding the entire body.

In view of such a circumstance, the present invention allows users tomore surely and more easily take images of a subject in a specific pose.

An information processing apparatus according to an embodiment of thepresent invention includes image pickup means capturing an image of asubject, extracting means extracting a human figure from the capturedimage of the subject captured by the image pickup means, estimatingmeans estimating a posture of the human figure extracted by theextracting means, categorizing means categorizing the posture of thehuman figure estimated by the estimating means into a previouslyprepared pose, and recording control means controlling recording of thecaptured image on the basis of the pose that the posture of the humanfigure is categorized into by the categorizing means.

The recording control means may control recording of the captured imagein a case where the pose that the posture of the human figure iscategorized into by the categorizing means is a previously decidedrecording pose for recording the captured image.

The recording control means may control recording of the captured imagein a case where a plurality of human figures are extracted by theextracting means and at least some of the poses that the postures of theplurality of human figures are categorized into by the categorizingmeans are the same recording pose.

The information processing apparatus may further include mode settingmeans setting an image capturing mode in accordance with the pose thatthe posture of the human figure is categorized into by the categorizingmeans. The recording control means may control recording of the capturedimage in the image capturing mode set by the mode setting means in acase where the pose is the recording pose.

The information processing apparatus may further include comparing meanscomparing the posture of the human figure estimated by the estimatingmeans between frames and shutter speed setting means setting shutterspeed in image capturing of the image pickup means in accordance with achange in the posture of the human figure between the frames compared bythe comparing means.

An information processing method according to an embodiment of thepresent invention is an image capturing method of an informationprocessing apparatus including image pickup means capturing an image ofa subject and includes the steps of extracting a human figure from thecaptured image of the subject captured by the image pickup means,estimating a posture of the human figure extracted in the processing ofextracting, categorizing the posture of the human figure estimated inthe processing of estimating into a previously prepared pose, andcontrolling recording of the captured image on the basis of the posethat the posture of the human figure is categorized into in processingof categorizing.

A program according to an embodiment of the present invention is aprogram causing a computer to execute image capturing processing of aninformation processing apparatus including image pickup means capturingan image of a subject. The program causes the computer to execute theprocessing including the steps of extracting a human figure from thecaptured image of the subject captured by the image pickup means,estimating a posture of the human figure extracted in the processing ofextracting, categorizing the posture of the human figure estimated inthe processing of estimating into a previously prepared pose, andcontrolling recording of the captured image on the basis of the posethat the posture of the human figure is categorized into in theprocessing of categorizing.

In accordance with an embodiment of the present invention, a humanfigure is extracted from a captured image of a subject. A posture of theextracted human figure is estimated. The estimated posture of the humanfigure is categorized into a previously prepared pose. Recording of thecaptured image is controlled on the basis of the pose that the postureof the human figure is categorized into.

An embodiment of the present invention allows users to more surely andmore easily take images of a subject in a specific pose.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of a functionalconfiguration of an image capturing apparatus serving as one embodimentof an information processing apparatus that the present invention isapplied to;

FIG. 2 is a flowchart describing image capturing processing;

FIG. 3 is a diagram describing an example of a human region;

FIG. 4 is a diagram describing posture estimation;

FIG. 5 is a diagram describing a learning dictionary;

FIG. 6 is a block diagram illustrating an example of another functionalconfiguration of an image capturing apparatus;

FIG. 7 is a flowchart describing image capturing processing of the imagecapturing apparatus illustrated in FIG. 6;

FIG. 8 is a block diagram illustrating an example of a still anotherfunctional configuration of an image capturing apparatus;

FIG. 9 is a flowchart describing image capturing processing of the imagecapturing apparatus illustrated in FIG. 8;

FIG. 10 is a block diagram illustrating an example of a still anotherfunctional configuration of an image capturing apparatus;

FIG. 11 is a flowchart describing image capturing processing of theimage capturing apparatus illustrated in FIG. 10; and

FIG. 12 is a block diagram illustrating an example of a hardwareconfiguration of a computer.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention will be described below withreference to the attached drawings. Meanwhile, the description will begiven in an order as follows:

1. First embodiment (an example case of one subject);

2. Second embodiment (an example case of a plurality of subjects);

3. Third embodiment (an example of setting an image capturing mode inaccordance with a pose); and

4. Fourth embodiment (an example of setting shutter speed in accordancewith movement of a subject).

<1. First Embodiment>

[About Example of Functional Configuration of Image Capturing Apparatus]

FIG. 1 illustrates an example of a functional configuration of an imagecapturing apparatus serving as an embodiment of an informationprocessing apparatus that the present invention is applied to.

An image capturing apparatus 11 illustrated in FIG. 1 may beapparatuses, such as a digital camera and an image-capturing-functionattached mobile phone.

The image capturing apparatus 11 illustrated in FIG. 1 includes anoptical unit 31, an image pickup unit 32, an image processing unit 33,an image capturing control unit 34, a recording control unit 35, arecording unit 36, a display control unit 37, and a display unit 38.

The optical unit 31 includes optical elements, such as a lens, an iris,and a mechanical shutter, and adjusts a focal position and exposure. Atthe time of image capturing, the optical unit 31 passes light incomingoutside of the image capturing apparatus 11 and supplies the light tothe image pickup unit 32.

The image pickup unit 32 includes a photoelectric element, such as acharge coupled device (CCD) or a complementary metal oxide semiconductor(CMOS) sensor. The image pickup unit 32 converts the incident light(i.e., a captured image) supplied through the optical unit 31 intoelectronic data with the photoelectric element thereof to generate imagedata of the captured image. The image pickup unit 32 supplies thegenerated image data to the image processing unit 33.

The image processing unit 33 performs predetermined image processing onthe image data supplied from the image pickup unit 32. The imageprocessing unit 33 supplies the processed image data to the recordingunit 36 through the recording control unit 35 to store the image dataand also supplies the image data to the display unit 38 through thedisplay control unit 37 to display an image thereof. Additionally, theimage processing unit 33 supplies the image capturing control unit 34with information for controlling image capturing performed in the imagecapturing apparatus 11 resulting from the image processing.

The image processing unit 33 includes a signal processor 51, a facedetector 52, a human region extractor 53, a posture estimator 54, a posecategorizer 55, and a pose determiner 56.

The signal processor 51 performs, on the image data supplied from theimage pickup unit 32, image processing, such as white balance adjustmentprocessing, de-mosaic processing, tone correction processing, gammacorrection processing, and YC conversion processing. The signalprocessor 51 supplies the processed image data (hereinafter, referred toas a captured image) to the face detector 52, the human region extractor53, the recording control unit 35, and the display control unit 37.

The face detector 52 detects a face from the image data (i.e., thecaptured image) supplied from the signal processor 51 and suppliesposition information representing a position of the detected face to thehuman region extractor 53.

The human region extractor 53 extracts a human region, i.e., a region ofa human figure in the captured image, on the basis of the positioninformation supplied from the face detector 52 and supplies theextracted human region to the posture estimator 54.

The posture estimator 54 estimates a posture of the human figure in thehuman region supplied from the human region extractor 53 and suppliesposture information representing the posture to the pose categorizer 55.

The pose categorizer 55 categorizes the posture of the human figure inthe captured image into one of poses prepared in advance on the basis ofthe posture information supplied from the posture estimator 54. The posecategorizer 55 supplies the pose determiner 56 with pose informationrepresenting the pose that the posture of the human figure in thecaptured image is categorized into.

The pose determiner 56 determines whether the pose represented by thepose information supplied from the pose categorizer 55 is apredetermined shutter pose for recording the captured image. If the poserepresented by the pose information supplied from the pose categorizer55 is the shutter pose, the pose determiner 56 supplies the imagecapturing control unit 34 with information for instructing recording ofthe captured image.

The image capturing control unit 34 controls the optical unit 31, theimage pickup unit 32, the recording control unit 35, and the signalprocessor 51, thereby realizing an image capturing function in the imagecapturing apparatus 11. The image capturing control unit 34 controlsrecording of the captured image in the recording unit 36 through therecording control unit 35 on the basis of the information supplied fromthe pose determiner 56.

The recording control unit 35 controls recording in the recording unit36 on the basis of the control of the image capturing control unit 34 torecord the captured image supplied from the signal processor 51 in therecording unit 36.

The display control unit 37 controls display on the display unit 38 todisplay the captured image supplied from the signal processor 51 on thedisplay unit 38.

[About Image Capturing Processing of Image Capturing Apparatus]

Image capturing processing of the image capturing apparatus 11illustrated in FIG. 1 will now be described with reference to aflowchart in FIG. 2. The image capturing processing illustrated in FIG.2 is executed every time the signal processor 51 performs the imageprocessing on the image data supplied from the image pickup unit 32 on aframe-by-frame basis.

In STEP S11, the face detector 52 detects a face of a human figure froma predetermined frame of the captured image (i.e., image data) suppliedfrom the signal processor 51. For example, the face detector 52 haslearned face images of faces in various directions, thereby detecting ahuman face from the captured image.

In STEP S12, the face detector 52 determines whether the face has beendetected. If it is determined in STEP S12 that the face has not beendetected, the process returns to STEP S11. The face detector 52 repeatsthe processing in STEPs S11 and S12 until a frame in which one face isdetected is supplied from the signal processor 51.

On the other hand, if it is determined in STEP S12 that the face hasbeen detected, the face detector 52 supplies the human region extractor53 with position information representing a position of the detectedface. Here, the position information may be, for example, coordinates ofupper left and lower right apices of a rectangular region detected as aregion of the face. Meanwhile, the position information at least allowsthe position of the face in the captured image to be specified betweenframes of the captured image.

In STEP S13, the human region extractor 53 extracts a human regionserving as a region of the human figure in the predetermined frame ofthe captured image on the basis of the position information suppliedfrom the face detector 52 and supplies the extracted human region to theposture estimator 54. More specifically, the human region extractor 53estimates a position of an upper half part of the human figure based onthe position of the face represented by the position informationsupplied from the face detector 52 and extracts a region of the face andthe estimated upper half part in the captured image as the human region.

In STEP S14, the posture estimator 54 estimates a posture of the humanfigure in the human region supplied from the human region extractor 53.

Posture estimation by the posture estimator 54 will now be describedwith reference to FIGS. 3 and 4.

FIG. 3 illustrates a human region extracted from a predetermined frameof a captured image. Referring to FIG. 3, a region of a face and anupper half part of a human figure thrusting their fist is extracted asthe human region.

Generally, a human body is a multi-jointed body that has a plurality ofjoints and changes in various shapes. A body segment part connecting thejoints can be considered as a rigid body. Thus, a human body can berepresented by determining an angle (hereinafter, referred to as a jointangle) between two body segments connected to one joint in a modelconstituted by connecting body segments each other through joints.

Accordingly, when the human region illustrated in FIG. 3 is extracted,the posture estimator 54 specifies each joint in the upper half part ofthe human figure thrusting their fist and determines the joint angleregarding each joint, thereby generating a three-dimensional human bodymodel illustrated in FIG. 4 and estimating a posture of the upper halfpart of the human figure.

The three-dimensional human body model illustrated in FIG. 4 isrepresented with a head H, joints J1 to J10, and body segmentsconnecting each joint. Referring to FIG. 4, the joint J1, the joint J2,the joint J3, the joint J4, the joint J5, the joint J6, the joint J7,the joint J8, the joint J9, and the joint J10 represent a neck, a rightshoulder, a right elbow, a right wrist, a left shoulder, a left elbow, aleft wrist, a lower back, a right groin, and a left groin, respectively,to correspond the upper half part of the human figure illustrated inFIG. 3.

The posture estimator 54 then supplies the pose categorizer 55 withthree-dimensional-space coordinates (hereinafter, referred to as jointcoordinates) of each joint of the three-dimensional human body modelgenerated in this way as the posture information representing theposture of the upper half part of the human figure.

Referring back to the flowchart in FIG. 2, in STEP S15, the posecategorizer 55 determines which of the poses registered in an internallyheld learning dictionary matches the posture of the human figure in thecaptured image on the basis of the posture information supplied from theposture estimator 54, thereby categorizing the estimated posture.

More specifically, as illustrated in FIG. 5, the learning dictionaryheld in the pose categorizer 55 stores poses, such as a pose ofthrusting one's fist, a pose of raising arms, a pose of raising an arm,a standing posture, in association with the joint coordinates of each ofthe above-described joints J1 to J10.

In accordance with FIG. 5, for example, the pose of thrusting one's fistis a posture for which the joints J1 to J10 are located at coordinates(xa1, ya1, za1), (xa2, ya2, za2), . . . , (xa10, ya10, za10) in thethree-dimensional space (e.g., a xyz space), respectively. Additionally,for example, the pose of raising arms is a posture for which the jointsJ1 to J10 are located at coordinates (xb1, yb1, zb1), (xb2, yb2, zb2), .. . , (xb10, yb10, zb10) in the three-dimensional space (e.g., the xyzspace), respectively.

In STEP S16, the pose categorizer 55 determines whether the postureestimated by the posture estimator 54 is a posture that can becategorized. More specifically, the pose categorizer 55 determineswhether joint coordinates close to each joint coordinates serving as theposture information supplied from the posture estimator 54 areregistered in the learning dictionary illustrated in FIG. 5.

If it is determined in STEP S16 that the posture estimated by theposture estimator 54 is not a posture that can be categorized, i.e., ifthe joint coordinates close to each joint coordinates serving as theposture information supplied from the posture estimator 54 are notregistered, the process returns to STEP S11 and the processing isexecuted on the next frame. Meanwhile, at this time, the posecategorizer 55 may supply the display control unit 37 with informationindicating that the estimated posture is not categorizable. In thiscase, the display control unit 37 causes the display unit 38 to displaythat the estimated posture is not categorizable.

On the other hand, if it is determined in STEP S16 that the postureestimated by the posture estimator 54 is a posture that can becategorized, i.e., if the joint coordinates close to each jointcoordinates serving as the posture information supplied from the postureestimator 54 are registered in the learning dictionary, the posecategorizer 55 supplies the pose determiner 56 with pose informationrepresenting the pose associated with the joint coordinates.

In STEP S17, the pose determiner 56 determines whether the poserepresented by the pose information supplied from the pose categorizer55 is a shutter pose.

The shutter pose is a pose set in advance by the user from the posesregistered in the learning dictionary held in the pose categorizer 55,for example.

If it is determined in STEP S17 that the pose represented by the poseinformation supplied from the pose categorizer 55 is not the shutterpose, the process returns to STEP S11 and the processing is executed onthe next frame.

On the other hand, if it is determined in STEP S17 that the poserepresented by the pose information supplied from the pose categorizer55 is the shutter pose, the pose determiner 56 supplies recordinginstruction information instructing recording of the captured image tothe image capturing control unit 34 together with shutter poseinformation representing the shutter pose.

In STEP S18, the image capturing control unit 34 causes the recordingcontrol unit 35 to control recording of the captured image in therecording unit 36 on the basis of the recording instruction informationsupplied from the pose determiner 56. At this time, the image capturingcontrol unit 34 causes the captured image to be recorded in therecording unit 36 together with the shutter pose information suppliedfrom the poser determiner 56.

In accordance with the foregoing processing, the posture of the humanfigure serving as the subject is estimated. If the estimated posture iscategorized into a predetermined pose and the predetermined pose is theshutter pose, the captured image can be recorded. That is, since thecaptured image is recorded if the pose taken by the subject isregistered in the learning dictionary and the pose is set as the shutterpose, an image taker can more surely and more easily take images of thesubject in the specific pose without being conscious of the pose takenby the subject.

Additionally, since the shutter pose information representing theshutter pose is recorded with the captured image, a user can categorizeand retrieve the captured image based on the shutter pose informationwhen organizing a plurality of captured images recorded in the recordingunit 36.

Although the image capturing processing in the case of one subject humanfigure has been described above, image capturing processing in a case ofa plurality of subject human figures will be described below.

<2. Second Embodiment>

[About Example of Another Functional Configuration of Image CapturingApparatus]

FIG. 6 illustrates an example of another functional configuration of animage capturing apparatus that the present invention is applied to.

An image capturing apparatus 111 illustrated in FIG. 6 includes anoptical unit 31, an image pickup unit 32, an image processing unit 33,an image capturing control unit 34, a recording control unit 35, arecording unit 36, a display control unit 37, and a display unit 38.Additionally, the image processing unit 33 illustrated in FIG. 6includes a signal processor 51, a human region extractor 53, a postureestimator 54, a face detector 151, a pose categorizer 152, and a posedeterminer 153.

Meanwhile, in the image capturing apparatus 111 illustrated in FIG. 6,the same name and the same reference sign are attached to aconfiguration having a function similar to that provided in the imagecapturing apparatus 11 illustrated in FIG. 1 and a description thereofis omitted accordingly.

That is, the image capturing apparatus 111 in FIG. 6 differs from theimage capturing apparatus 11 in FIG. 1 in that the face detector 151,the pose categorizer 152, and the pose determiner 153 are provided inthe image processing unit 33 instead of the face detector 52, the posecategorizer 55, and the pose determiner 56, respectively.

The face detector 151 detects a plurality of faces from image data(i.e., a captured image) supplied from the signal processor 51 andsupplies the human region extractor 53 with position informationrepresenting positions of the detected faces. Additionally, the facedetector 151 supplies the pose categorizer 152 with informationrepresenting the number of detected faces.

The pose categorizer 152 categorizes postures of a plurality of humanfigures in the captured image into one of poses prepared in advance onthe basis of posture information supplied from the posture estimator 54.The pose categorizer 152 supplies the pose determiner 153 with poseinformation representing poses that the postures of the plurality ofhuman figures in the captured image are categorized into.

The pose determiner 153 determines whether the plurality of posesrepresented by the pose information supplied from the pose categorizer152 are the same shutter pose. If the plurality of poses represented bythe pose information supplied from the pose categorizer 152 are the sameshutter pose, the pose determiner 153 supplies information instructingrecording of the captured image to the image capturing control unit 34.

[About Image Capturing Processing of Image Capturing Apparatus]

Image capturing processing of the image capturing apparatus 111illustrated FIG. 6 will now be described with reference to a flowchartin FIG. 7. The image capturing processing illustrate in FIG. 7 isexecuted every time the signal processor 51 performs image processing onimage data supplied from the image pickup unit 32 on a frame-by-framebasis.

In STEP S111, the face detector 151 detects a plurality of faces ofhuman figures from a predetermined frame of a captured image (i.e.,image data) supplied from the signal processor 51.

In STEP S112, the face detector 151 determines whether the faces havebeen detected. If it is determined in STEP S112 that no face has beendetected, the process returns to STEP S111 and the face detector 151repeats the processing in STEPs 5111 and 5112 until a frame in which thefaces are detected is supplied from the signal processor 51.

On the other hand, if it is determined in STEP S112 that the faces havebeen detected, the face detector 151 supplies the human region extractor53 with position information representing positions of one or moredetected faces. Additionally, the face detector 151 supplies the posecategorizer 152 with information representing the number of detectedfaces.

In STEP S113, the human region extractor 53 extracts a human regionserving as a region of the human figure in the predetermined frame ofthe captured image on the basis of the position information suppliedfrom the face detector 52 and supplies the extracted human region to theposture estimator 54. Here, as many human regions as the number ofdetected faces are extracted.

In STEP S114, the posture estimator 54 estimates a posture of the humanfigure in the human region supplied from the human region extractor 53as described with reference to FIGS. 3 and 4. The posture estimator 54supplies the pose categorizer 152 with as many pieces of postureinformation representing the estimated posture as the number of facesdetected by the face detector 151.

In STEP S115, the pose categorizer 152 determines whether each of thepostures of the plurality of human figures in the captured image matchesone of poses registered in a learning dictionary held therein on thebasis of the posture information supplied from the posture estimator 54,thereby categorizing the postures estimated for the plurality of humanfigures.

In STEP S116, the pose categorizer 152 determines whether the posturesestimated by the posture estimator 54 for the plurality of human figuresare postures that can be categorized on the basis of the informationrepresenting the number of faces supplied from the face detector 151.

If it is determined in STEP S116 that some of the postures estimated bythe posture estimator 54 for the plurality of human figures are not thepostures that can be categorized, i.e., if the number of categorizablepostures is less than the number of faces represented by the informationsupplied from the face detector 151, the process returns to STEP S111and the processing is performed on the next frame. Additionally, at thistime, the pose categorizer 152 may supply the display control unit 37with information indicating that some of the postures estimated for theplurality of human figures are not categorizable. In this case, thedisplay control unit 37 causes the display unit 38 to display that someof the estimated postures are not categorizable.

On the other hand, if it is determined in STEP S116 that the posturesestimated by the posture estimator 54 are postures that can becategorized, i.e., if the number of categorizable postures matches thenumber of faces represented by the information supplied from the facedetector 151, the pose categorizer 152 supplies the pose determiner 153with pose information representing a pose that each posture iscategorized into.

In STEP S117, the pose determiner 153 determines whether the pluralityof poses represented by the pose information supplied from the posecategorizer 152 are the same shutter pose.

If it is determined in STEP S117 that the plurality of poses representedby the pose information supplied from the pose categorizer 152 are notthe same shutter pose, i.e., if one of the plurality of poses is ashutter pose different from the others or is not the set shutter pose,the process returns to STEP S111 and the processing is performed on thenext frame.

On the other hand, if it is determined in STEP S117 that the pluralityof poses represented by the pose information supplied from the posecategorizer 152 are the same shutter pose, the pose determiner 153supplies the image capturing control unit 34 with recording instructioninformation instructing recording of the captured image and shutter poseinformation representing the shutter pose.

In STEP S118, the image capturing control unit 34 causes the recordingcontrol unit 35 to control recording of the captured image in therecording unit 36 on the basis of the recording instruction informationsupplied from the pose determiner 153. At this time, the image capturingcontrol unit 34 causes the recording unit 36 to record the capturedimage together with the shutter pose information supplied from the posedeterminer 153.

In accordance with the foregoing processing, the postures of theplurality of human figures serving as the subjects are estimated. If thepostures estimated for the plurality of human figures are categorizedinto predetermined poses and the poses are the same shutter pose, thecaptured image can be recorded. That is, the captured image is recordedif the poses taken by the plurality of subjects are registered in thelearning dictionary and the poses are set as the shutter poses and ifthe poses taken by the plurality of subjects are the same. Thus, animage taker can more surely and more easily take images of the pluralityof subjects in the poses without being conscious of the poses taken bythe plurality of subjects when the poses taken by the plurality ofsubjects match.

Additionally, since the shutter pose information representing theshutter pose is recorded with the captured image, a user can categorizeand retrieve the captured image based on the shutter pose informationwhen organizing a plurality of captured images recorded in the recordingunit 36.

Meanwhile, in the foregoing description, the pose categorizer 152supplies the pose information to the pose determiner 153 when the numberof categorizable postures matches the number of detected faces in STEPS116. However, for example, the pose information may be supplied to thepose determiner 153 when the number of categorizable postures matches avalue resulting from subtraction of a predetermined value, such as 2 or3, from the number of detected faces.

In addition, the pose determiner 153 supplies the recording instructioninformation to the image capturing control unit 34 when the plurality ofposes are the same shutter pose in STEP S117. However, when it isdetermined that, for example, 80% of the plurality of poses are the sameshutter pose, the recording instruction information may be supplied tothe image capturing control unit 34.

Accordingly, if the poses taken by the plurality of subjects are not thesame but the poses taken by the plurality of subjects match to somedegree, the captured image of the subjects can be recorded.

Although the processing for recording the captured image of the subjectsin accordance with the poses taken by the subjects has been describedabove, the captured image of the subject can be recorded after an imagecapturing mode is set in accordance with the pose taken by the subject.

<3. Third Embodiment>

[About Example of Another Functional Configuration of Image CapturingApparatus]

FIG. 8 illustrates an example of a functional configuration of an imagecapturing apparatus that records a captured image of a subject aftersetting an image capturing mode in accordance with a pose taken by thesubject.

An image capturing apparatus 211 illustrated in FIG. 8 includes anoptical unit 31, an image pickup unit 32, an image processing unit 33, arecording control unit 35, a recording unit 36, a display control unit37, a display unit 38, and an image capturing control unit 231.

Meanwhile, in the image capturing apparatus 211 illustrated in FIG. 8,the same name and the same reference sign are attached to aconfiguration having a function similar to that provided in the imagecapturing apparatus 11 illustrated in FIG. 1 and a description thereofis omitted accordingly.

That is, the image capturing apparatus 211 in FIG. 8 differs from theimage capturing apparatus 11 in FIG. 1 in that the image capturingcontrol unit 231 is provided instead of the image capturing control unit34.

Additionally, a pose categorizer 55 illustrated in FIG. 8 supplies apose determiner 56 and the image capturing control unit 231 with poseinformation representing a pose that a posture of a human figure in acaptured image is categorized into.

The image capturing control unit 231 has a function similar to that ofthe image capturing control unit 34 illustrated in FIG. 1 and alsoincludes an image capturing mode setter 231 a. The image capturing modesetter 231 a sets an image capturing mode on the basis of the poseinformation supplied from the pose categorizer 55 and suppliesinformation representing the set image capturing mode to the opticalunit 31, the image pickup unit 32, and a signal processor 51.

[About Image Capturing Processing of Image Capturing Apparatus]

Image capturing processing of the image capturing apparatus 211illustrated in FIG. 8 will now be described with reference to aflowchart illustrated in FIG. 9. Meanwhile, since processing in STEPsS211 to S216, S218, and S219 in the flowchart in FIG. 9 is similar toprocessing in STEPs S11 to 18 having been described with reference tothe flowchart in FIG. 2, a description thereof is omitted.

Additionally, if it is determined in STEP S216 that a posture estimatedby a posture estimator 54 is a posture that can be categorized, the posecategorizer 55 supplies the pose determiner 56 and the image capturingcontrol unit 231 with pose information representing a pose that theposture is categorized into.

In STEP S217, the image capturing mode setter 231 a of the imagecapturing control unit 231 sets an image capturing mode corresponding tothe pose represented by the pose information on the basis of the poseinformation supplied from the pose categorizer 55.

The image capturing mode setter 231 a holds correspondence informationassociating various poses with scene modes, serving as the imagecapturing modes, for performing image capturing corresponding to anenvironment in which the subject exits. The scene modes include, forexample, a human mode for enhancing a human figure so that a skin coloris imaged beautifully, a sport mode for capturing a subject moving fast,a landscape mode for focusing in the distance so that the image iscaptured sharply, a self shooting mode for correcting motion blurring sothat a skin color is imaged beautifully.

Additionally, the pose and the scene mode are associated in thecorrespondence information in a manner, such as a pose of thrustingone's fist and the sport mode, and a pose of raising arms and thelandscape mode.

The image capturing mode setter 231 a supplies the optical unit 31, theimage pickup unit 32, and the signal processor 51 with mode informationrepresenting the scene mode associated with the pose represented by thepose information supplied from the pose categorizer 55 on the basis ofthe correspondence information.

In this way, image capturing parameters are adjusted in the optical unit31 and the image pickup unit 32 and predetermined image processing isperformed in the signal processor 51 so that a captured imagecorresponding to the scene mode represented by the mode information isobtained.

For example, when a pose of extending the right hand towards the imagecapturing apparatus 211 is registered in a learning dictionary and thepose is set as a shutter pose, shutter timing can be delayed by inadvance preparing the correspondence information associating the posewith the self capturing mode.

In accordance with the foregoing processing, the posture of the humanfigure serving as the subject is estimated. If the posture estimated forthe human figure is categorized into a predetermined pose, the capturedimage can be recorded in the image capturing mode (i.e., the scene mode)corresponding to the pose. That is, if the pose taken by the subject isregistered in the learning dictionary and the pose is set as the shutterpose, the captured image is recorded in the scene mode corresponding tothe pose. Thus, an image taker can more surely and more easily takeimages of the subject in the pose without being conscious of the posetaken by the subject and without setting the scene mode that isgenerally manually set through an operation, such as a dial operation.

Although the processing for recording the captured image of the subjectafter setting the image capturing mode in accordance with the pose takenby the subject has been described above, the captured image of thesubject can be recorded in accordance with the pose taken by the subjectafter setting shutter speed on the basis of an estimated posture change.

<4. Fourth Embodiment>

[About Example of Another Functional Configuration of Image CapturingApparatus]

FIG. 10 illustrates an example of a functional configuration of an imagecapturing apparatus that records a captured image of a subject inaccordance with a pose taken by the subject after setting shutter speedon the basis of an estimated posture change.

An image capturing apparatus 311 illustrated in FIG. 10 includes anoptical unit 31, an image pickup unit 32, an image processing unit 33, arecording control unit 35, a recording unit 36, a display control unit37, a display unit 38, and an image capturing control unit 331.Additionally, the image processing unit 33 illustrated in FIG. 10includes a signal processor 51, a face detector 52, a human regionextractor 53, a posture estimator 54, a pose categorizer 55, a posedeterminer 56, a posture information holder 351, and a postureinformation comparator 352.

Meanwhile, in the image capturing apparatus 311 illustrated in FIG. 10,the same name and the same reference sign are attached to aconfiguration having a function similar to that provided in the imagecapturing apparatus 11 illustrated in FIG. 1 and a description thereofis omitted accordingly.

That is, the image capturing apparatus 311 in FIG. 10 differs from theimage capturing apparatus 11 in FIG. 1 in that the image capturingcontrol unit 331 is provided instead of the image capturing control unit34 and the posture information holder 351 and the posture informationcomparator 352 are newly provided in the image processing unit 33.

Meanwhile, the posture estimator 54 illustrated in FIG. 10 estimates aposture of a human figure in a human region supplied from the humanregion extractor 53 and supplies posture information representing theposture to the pose categorizer 55, the posture information holder 351,and the posture information comparator 352.

The posture information holder 351 holds (stores) the postureinformation for one frame supplied from the posture estimator 54 andsupplies the posture information comparator 352 with posture informationdelayed by one frame.

The posture information comparator 352 compares the posture informationsupplied from the posture estimator 54 with the posture information ofthe last frame supplied from the posture information holder 351 andsupplies the image capturing control unit 331 with a result of thecomparison.

The image capturing control unit 331 has a function similar to that ofthe image capturing control unit 34 illustrated in FIG. 1 and alsoincludes a shutter speed setter 331 a. The shutter speed setter 331 asets shutter speed on the basis of the comparison result of theinterframe posture information supplied from the posture informationcomparator 352 and supplies the optical unit 31 with informationrepresenting the set shutter speed.

[About Image Capturing Processing of Image Capturing Apparatus]

Image capturing processing of the image capturing apparatus 311illustrated in FIG. 10 will now be described with reference to aflowchart illustrated in FIG. 11. Meanwhile, since processing in STEPsS311 to S314 and S318 to S321 in the flowchart in FIG. 11 is similar tothe processing in STEPs S11 to S18 having been described with referenceto the flowchart in FIG. 2, a description thereof is omitted.

Meanwhile, in STEP S314, the posture estimator 54 estimates a posture ofa human figure in a human region supplied from the human regionextractor 53 and supplies the pose categorizer 55, the postureinformation holder 351, and the posture information comparator 352 withposture information representing the estimated posture.

In STEP S315, the posture information holder 351 stores the postureinformation for one frame supplied from the posture estimator 54 andsupplies the posture information comparator 352 with the stored postureinformation of the last frame.

In STEP S316, the posture information comparator 352 compares theposture information supplied from the posture estimator 54 with theposture information of the last frame supplied from the postureinformation holder 351. More specifically, the posture informationcomparator 352 compares joint coordinate serving as the postureinformation of the current frame with joint coordinates serving as theposture information of the last frame, calculates an average value of anamount of movement of each of corresponding joints in the frames, forexample, and supplies the average value to the image capturing controlunit 331.

In STEP S317, the shutter speed setter 331 a of the image capturingcontrol unit 331 sets shutter speed on the basis of the comparisonresult of the interframe posture information supplied from the postureinformation comparator 352. More specifically, the shutter speed setter331 a sets the shutter speed in accordance with the average value of theamount of movement of the respective joints in the frames, i.e., speedof a change in the posture of the human figure, supplied from theposture information comparator 352. That is, the shutter speed setter331 a increases the shutter speed if the average value of the amount ofmovement of the respective joints in the frames is large, whereas itdecreases the shutter speed if the average value of the amount ofmovement of the respective joints in the frames is small. Shutter speedinformation representing the set shutter speed is supplied to theoptical unit 31.

In this way, exposure time is adjusted in the optical unit 31 so that acaptured image corresponding to the speed of the change in the postureof the human figure is obtained.

In accordance with the foregoing processing, the posture of the humanfigure serving as the subject is estimated. If the posture estimated forthe human figure is categorized into a predetermined pose, the capturedimage can be recorded in the shutter speed corresponding to the changein the estimated posture. That is, if the pose taken by the subject isregistered in a learning dictionary and the pose is set as a shutterpose, the captured image is recorded in the shutter speed correspondingto the movement of the subject performed until the subject takes thepose. Thus, an image taker can more surely and more easily take imagesof the subject taking the pose without being conscious of the pose takenby the subject and the movement of the subject up to that point.

Additionally, the image capturing processing in the case where thenumber of human figures serving as the subject is one has been describedin the flowcharts illustrated in FIG. 9 and FIG. 11. However, imagecapturing processing in a case of a plurality of subjects can berealized by combining the image capturing apparatus 211 (FIG. 8) and theimage capturing apparatus 311 (FIG. 10) with the image capturingapparatus 111 illustrated in FIG. 6.

The above-described series of processing may be executed by hardware orsoftware. When the series of processing is executed by software, aprogram constituting the software is installed, from a program recordingmedium, in a computer embedded in dedicated hardware or, for example, ageneral-purpose personal computer or the like capable of executingvarious functions by installing various programs therein.

FIG. 12 is a block diagram illustrating an example of a hardwareconfiguration of a computer executing the above-described series ofprocessing by a program.

In the computer, a central processing unit (CPU) 901, a read only memory(ROM) 902, and a random access memory (RAM) 903 are connected to eachother through a bus 904.

Furthermore, an input/output (I/O) interface 905 is connected to the bus904. The I/O interface 905 is connected to an input unit 906, such as akeyboard, a mouse, and a microphone, an output unit 907, such as adisplay and a speaker, a storage unit 908, such as a hard disk and anonvolatile memory, a communication unit 909, such as a networkinterface, and a drive 910 driving a removable medium 911, such as amagnetic disk, an optical disc, a magneto-optical disk, or asemiconductor memory.

In the computer constituted in the foregoing manner, the CPU 901 loadsand executes, for example, a program stored in the storage unit 908 tothe RAM 903 through the I/O interface 905 and the bus 904, therebyexecuting the above-described series of processing.

The program executed by the computer (the CPU 901) is provided afterbeing recorded on the removable medium 911 serving as a package medium,such as a magnetic disk (including a flexible disk), an optical disc(such as a compact disc-read only memory (CD-ROM) or a digital versatiledisk (DVD)), a magneto-optical disk, or a semiconductor memory, or via awired or wireless transmission medium, such as a local area network, theInternet, or digital satellite broadcasting.

The program can be then installed in the storage unit 908 through theI/O interface 905 by equipping the drive 910 with the removable medium911. Additionally, the program can be installed in the storage unit 908by receiving the program with the communication unit 909 via a wired orwireless transmission medium. Alternatively, the program can bepreinstalled in the ROM 902 or the storage unit 908.

Meanwhile, the program executed by the computer may be a program ofwhich processing is chronologically performed in an order described inthis specification or may be a program of which processing is performedin parallel or at necessary timing, such as when a call is performed.

Additionally, embodiments of the present invention are not limited tothe above-described embodiments and can be variously modified within ascope not departing from the spirit of the present invention.

The present application contains subject matter related to thatdisclosed in Japanese Priority Patent Application JP 2010-076304 filedin the Japan Patent Office on Mar. 29, 2010, the entire contents ofwhich are hereby incorporated by reference.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. An information processing apparatus, comprising: image pickup meanscapturing an image of a subject; extracting means extracting a humanfigure from the captured image of the subject captured by the imagepickup means; estimating means estimating a posture of the human figureextracted by the extracting means; categorizing means categorizing theposture of the human figure estimated by the estimating means into apreviously prepared pose; and recording control means controllingrecording of the captured image on the basis of the pose that theposture of the human figure is categorized into by the categorizingmeans.
 2. The information processing apparatus according to claim 1,wherein the recording control means controls recording of the capturedimage in a case where the pose that the posture of the human figure iscategorized into by the categorizing means is a previously decidedrecording pose for recording the captured image.
 3. The informationprocessing apparatus according to claim 2, wherein the recording controlmeans controls recording of the captured image in a case where aplurality of human figures are extracted by the extracting means and atleast some of the poses that the postures of the plurality of humanfigures are categorized into by the categorizing means are the samerecording pose.
 4. The information processing apparatus according toclaim 2, further comprising: mode setting means setting an imagecapturing mode in accordance with the pose that the posture of the humanfigure is categorized into by the categorizing means, wherein therecording control means controls recording of the captured image in theimage capturing mode set by the mode setting means in a case where thepose is the recording pose.
 5. The information processing apparatusaccording to claim 2, further comprising: comparing means comparing theposture of the human figure estimated by the estimating means betweenframes; and shutter speed setting means setting shutter speed in imagecapturing of the image pickup means in accordance with a change in theposture of the human figure between the frames compared by the comparingmeans.
 6. An information processing method of an information processingapparatus including image pickup means capturing an image of a subject,the information processing method comprising the steps of: extracting ahuman figure from the captured image of the subject captured by theimage pickup means; estimating a posture of the human figure extractedin the processing of extracting; categorizing the posture of the humanfigure estimated in the processing of estimating into a previouslyprepared pose; and controlling recording of the captured image on thebasis of the pose that the posture of the human figure is categorizedinto in the processing of categorizing.
 7. A program causing a computerto execute image capturing processing of an information processingapparatus including image pickup means capturing an image of a subject,the image capturing processing comprising the steps of: extracting ahuman figure from the captured image of the subject captured by theimage pickup means; estimating a posture of the human figure extractedin the processing of extracting; categorizing the posture of the humanfigure estimated in the processing of estimating into a previouslyprepared pose; and controlling recording of the captured image on thebasis of the pose that the posture of the human figure is categorizedinto in the processing of categorizing.
 8. An information processingapparatus, comprising: an image pickup unit capturing an image of asubject; an extracting unit extracting a human figure from the capturedimage of the subject captured by the image pickup unit; an estimatingunit estimating a posture of the human figure extracted by theextracting unit; a categorizing unit categorizing the posture of thehuman figure estimated by the estimating unit into a previously preparedpose; and a recording control unit controlling recording of the capturedimage on the basis of the pose that the posture of the human figure iscategorized into by the categorizing unit.